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Background / Context: 

Many community college students do not persist in eollege. Among students who intend to earn 
a eredential or to transfer to a four-year institution, only half meet that goal within six years. 
Many eommunity eollege students are referred to developmental edueation due to inadequate 
preparation for college-level work. For those students referred to developmental math, 
completing the required developmental math sequenee often poses a significant barrier, as does 
passing even the lowest level of eollege-level math, whieh is often a requirement for proeeeding 
toward a degree. Many eolleges a trying different methods to improve the sueeess rate in math. 
This study is a rigorous evaluation of a low-eost intervention aimed at improving students’ 
performance in developmental math and the lowest level of college math. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

This research rigorously evaluates whether a low-cost intervention can improve students’ 
performance in developmental math. The “Beacon Mentoring Program” was developed at South 
Texas College by professors, administrators, and staff at the college. Surveys of students 
revealed that many did not have someone on campus whom they felt they could turn to for 
answers to questions. Further, although the college provided many services, like a tutoring 
center, many students said they had never visited. The “Beacon Mentoring” program was 
constructed to bring critical information to students about how to manage college, and, thus, to 
improve college outcomes. 

Setting: 

Description of the research location. 

South Texas College is located in McAllen, Texas, near the border with Mexico. It serves over 
27,000 students, ninety-five percent of whom are Hispanic. South Texas College was on of the 
first community colleges to be chosen to participate in the Lumina Foundation’s Achieving the 
Dream Initiative. Under the auspices of Achieving the Dream, it chose to focus on improving 
performance in developmental math and other gatekeeper courses. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features or characteristics. 

Study participants were students enrolled in either developmental math or the first level of 
college math (pre-algebra, a gatekeeper course). There were 2,165 students in the study, over 
95% of whom are Hispanic. Fifty-seven percent of the students are female. About 70 percent are 
younger than 25 years old. Forty-six percent received some form of financial aid. Finally, 47.3 
percent were enrolled in school part-time. 



Intervention / Program / Practice: 
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The Beacon Mentoring program had three simple components: a) deliver information about on- 
campus services directly to students in the classroom, rather than waiting for students to seek out 
such information; b) provide students with a “go to” person on campus who knows who they are 
and can “broker” assistance, and c) improve communication between faculty and student 
services professionals to help students before they get into academic trouble. Mentors met with 
their assigned math class 3 or 4 times during the semester for 5 to 10 minutes. They encouraged 
students to email or meet with them in person if they had questions or problems. Finally, they 
collaborated with instructors to contact students at risk of failing the course. 

Research Design: 

Description of research design (e.g., qualitative case study, quasi-experimental design, secondary analysis, analytic 
essay, randomized field trial). 

This study is randomized controlled trail of the Beacon Mentoring program. However, 
randomization is at the level of the classroom, not the student level. In January 2008, over 2,000 
students enrolled as they normally would in 83 sections of three math courses (two 
developmental, and one college-level). The faculty assigned to teach these sections had agreed to 
participate in the study and understood that each section they taught had about a 50 percent 
chance of being assigned a Mentor. During the first week of class, MDRC randomly assigned 
these 83 sections o either the program group (where the section was assigned a Mentor) or the 
control group (where no Mentor was assigned). 

This research design has advantages over random assignment at the individual level in that it is 
easier to implement than typical student level random assignment. Multiple sections taught by 
the same instructor were evenly divided between the program and control groups in order to 
control for teacher effects; day and evening classes were also evenly divided between program 
and control groups. Analysis of the data show that the random assignment led to program and 
control groups that are balanced in terms of observable characteristics (Table 1). 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

Most of the data analysis uses student level administrative records on transcripts and use of 
campus services. Survey data and qualitative focus group data were also collected from students 
and Mentors. The administrative records contain information on all courses taken, grades and 
withdrawals, total credits attempted and earned (and which classes these were), student 
persistence past the program semester, and the number of times students accessed student 
services. 

Regression analysis is used to calculate the program impacts on performance in math classes, 
overall college performance, persistence, and use of on-campus services. The regression 
analysis controls for random assignment pool fixed effects, and student characteristics. 
Controlling for student characteristics does not change the estimated impact, as expected since 
the program and control groups are balanced in terms of observable characteristics, but does 
improve the precision of the estimates. Standard errors are clustered at the math section level. 



2011 SREE Conference Abstract Template 



2 




Findings / Results: 

Description of the main findings with specific details 

Overall, the Beacon Mentoring program significantly increased the fraction of students who 
accessed the on-campus tutoring center. There was also a statistically significant decrease in 
withdrawal rates from math classes. However, the program had no statistically significant 
impact on math class pass rates, credits earned, nor on persistence in college. (Table 2, Table 3) 

However, turning to two important subgroups, part-time students and those enrolled in 
developmental math, the study finds statistically significant impacts on math class performance. 
For part-time students, the percent of students passing their math class increases by 6.6 
percentage points for program students over controls. Final exam scores for part-time students in 
developmental math also increase. Credits earned are also significantly higher for part-time 
students in the Beacon program (Table 4). For students enrolled in developmental math, the 
probability of withdrawing from math class declines, and total credits earned rises for program 
students over controls (Table 5). 
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Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

For students overall, the program impaets are few and modest. However, when part-time 
students and students in the two lowest level math elasses are eonsidered, there are statistieally 
signifieant program impaets. Given that the intervention was “light touch” and low-cost, any 
impact is important. 

There are three important conclusions from this study. First, whole-class random assignment in 
college settings is both feasible and efficient. Second, linking services to the classroom is 
potentially a powerful way to ensure that students know the services that are available to them. 
Finally, since the program was very successful at increasing students’ use of on-campus 
academic supports (tutoring), and yet performance did not increase for students overall, it 
suggests that the type and quality of academic support offered in student success centers should 
be better understood. 



2011 SREE Conference Abstract Template 




Appendices 

Not included in page count. 



Appendix A, References 

References are to be in APA version 6 format. 



2011 SREE Conference Abstract Template 



A-1 




Appendix B, Tables and Figures 

Not included in page count. 



Table 1; Program Control Differences in Characteristics 



d 


Program Group 


Control Group 


% Female 


57.5 


57.6 


% Under 24 Years 


70.3 


71.1 


% Evening 


16.6 


17.9 


GPAinH.S. ‘ 


0.8 


0.8 


Math Placement Test: 






Top 25% ^ 


23.3 


26.0 


Bottom 25%^ 


24.8 


24.1 
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Difference 


Standard Error 


0.0 


2.1 


-0.8 


2.0 


-1.3 


1.6 


0.0 


0.0 






-2.7 


1.9 


0.8 


1.0 
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' "able 2: Overall Sample 



Outcome 


Program 

Group 


Control 

Group Diffe 


ren ce 


Stan dard 
Error 


Math course pass rate (%) 


54.81 


53.38 


1.43 


1.86 


Final exa m score “ 


51.70 


50.50 


1.21 


1.31 


Percent days absent‘d (%) 


19.09 


21.89 


-2.81 


* 1.50 


Number of days attended" 


20.27 


19.86 


0.41 


0.63 


W ith draw from math cours e (%) 


15.38 


18.20 


-2.82 


* 1.52 


Credits attempted 


10.37 


10.53 


-0.16 


** 0.07 


Credits earned 


6.56 


6.49 


0.07 


0.16 


GPA 


1 .97 


2.01 


-0.03 


0.04 


Postprogram persistence (%) 

Registered spring 2008 and fall 2008 


58.31 


59.01 


-0.70 


2.02 


Sample size (total students = 2165) 


1067 


1098 






Sections represented (total = 83) 


41 


42 
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Table 3 



Pf rcentagf lllio Ever Visited llie Center for Learning Excellence, by Subgroup 



Ever Visited Center f o) 


Sample Program Control 
Size Group Group 


Difference 


Difference 
Standard Benveen 
Error Subgroups 


M students 


m 25,5 19.8 


m 


2,0 NA 


Full time (12+ credits) 
Part time (less tlian 12) 


U40 25,6 20.2 

1,025 25.3 19.3 


5,4** 

6,0** 


2.7 

2.4 


EnroOed in developmental matli 
EnroOed in college-level matli 


1.249 21.5 112 

916 31.0 23.2 


4.3* 

7.8** 


rr 

3.5 


Sample size (total students = 2 Jd)) 


1,067 1098 






Sections represented (total =8]) 


41 42 
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able 4; Program Impacts for Part-time Students 







Part-time 


(less th an 1 2 h ours) 




Program 


Control 




Stan dard 


Outcome 


Group 


Group 


Difference 


Error 


Math course pass rate (%) 


57.62 


51.07 


6.55 ** 


2.55 


Final exam score ^ 


54.2 3 


50.87 


3.37 * 


1.93 


Percent days absent‘d (%) 


18.62 


22.54 


-3.92 


2.56 


Number of days attended 


20.31 


19.69 


0.61 


0.81 


W ithdraw from math course (%) 


14.11 


20.43 


-6.32 *** 


2.23 


Credits attempted 


7.68 


7.91 


-0.23 * 


0.13 


Credits earned 


4.92 


4.61 


0.31 * 


0.18 


GPA 


2.00 


1.96 


0.04 


0.06 


Postprogram persistence (%) 










Registered spring 2008 and fall 2008 


54.10 


54.94 


-0.84 


2.96 


Stu dents 


497 


528 






Sections represented (total = 83) 


41 


42 
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Table 5; Program Impacts for Developmental Math Students 



Outcome 


Students in Developmental Classes 


Program Control 
Group Group Difference 


Standard 

Erroi 


Math course pass rate (%) 


50.70 


47.62 


3.09 


2.61 


Withdraw from math course (%) 


11.79 


17.99 


-6.21 *** 


2.10 


Credits attempted 


9.88 


9.99 


-0.11 


0.09 


Qedits earned 


6.09 


5.55 


0.54 *** 


0.20 


GPA 


1.83 


1.78 


0.05 


0.05 


Postprogram persistence (%) 










Registered spring 2008 and fall 2008 


56.98 


55.91 


1.06 


2.67 


Students 


624 


625 






Sections 


26 


26 
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